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(57) A picture quality assessment method based up- 
on spatial location with or without subsampling within 
the picture divides the video picture into a plurality of 
arb,trarily sized, geometric segments. A segment 
weighting map is generated that provides one value for 
"safe picture" segments of the video picture and another 
value for the other segments. Optionally one or more of 
the segments may be subsampled to reduce processing 
time for realtime applications. The video picture on a 
segment basis is compared with a corresponding test 
video picture using a picture quality assessment algo- 
rithm to generate a quality error value for each segment 
The quality error values are weighted according to the 
segment weighting map, and on an error display the 
weighted quality error values are displayed. 



location with or without subsampling 



20 



2 >2 



—•CUT] 



2S 




CAPTURE/ANALYSIS <OSP> 



MEAS 
OUT 



601 

IN 



28 



1 






TEST 
OUT 


j PCI ] 


SDRAM 






j GENERATOR (DSP) 



30 

[ 



u 



24 



DE- 
SERIAL 



ENCODER 



— Ienccde 



GEN 
LOCK 



32 

\ 



COMP 



VI0 
OUT 



SERIAL 



-•-jOECODE 
T 



DECODER 



I FILTER 
A/0 



SYNC 
REr" 



601 
CUT 



VXD 



FIC.1 



BEST AVAILABLE GOPV 



Printed by Jouvo, 75001 PARIS (FR) 



EP 0 977 446 A3 



European Patent 
Office 



EUROPEAN SEARCH REPORT 



Application Number 

EP 99 30 5732 



DOCUMENTS CONS IDERED TO BE RELEVANT 



Category; 



Citation of document with indication, where appropriate. 
of relevant passages 



Relevant 
to claim 



CLASSIFICATION OF THE 
APPLICATION (im.C I.7) 



: ARDITO M ET AL: "CORRELATION BETWEEN 1,2 
I OBJECTIVE AND SUBJECTIVE MEASUREMENTS FOR 
| VIDEOCOMPRESSED SYSTEMS" 

SMPTE JOURNAL, US, SM PTE INC. SCARSDALE, 

N.Y, 

vol . 105, no. 12, 

1 December 1996 (1996-12-01), pages 
768-773, XP000643746 
ISSN: 0036-1682 
* page 772, middle column * 



H04N17/00 



TECHNICAL FIELDS 
SEARCHED <lm.CL7> 

H04N 



S3 1 

3 



The present search report has been drawn up lor all claims 

1 CatBO< 'campieiton oi the search - 

I 14 August 2QG1 



Place ot searcn 

THE HAGUE 



Yvonnet, J 



CATEGORY OF CITED COCUMENTS 

X ; particularly relevant il taken atone 

V : particularly relevant il comoined w4h another 

document of the same caieqcty 
A : technological baekorejnd 
O non-written disclosure 
P : intet mediate docutneni 



T : theory or principle underlying the invention 
£ : eaiiier patent document, but publisned on, or 

ar*er the fi ing date 
D : document cued in tne application 
'. : document cited for ether reasons 



ft : iremher of the sarre yatent lamiiy. corresponding 
documem 



BEST AVAILABLE COPV 



2 



(19) 



J 



Europaisches Patentamt 
European Patent Office 
Office europeen des brevets 




(12) 



(43) Date of publication: 

02.02.2000 Bulletin 2000/05 



(H) EP 0 977 446 A2 

EUROPEAN PATENT APPLICATION 

(51) Intel/: H04N 17/00 



CM 
< 
<£> 

o 



(21) Application number: 99305732.2 

(22) Date of filing: 20.07.1999 

(84) Designated Contracting States: 

AT BE CH CY DE DK ES Fl FR GB GR IE IT LI LU 
MCNLPTSE 

Designated Extension States: 
AL LT LV MK RO SI 

(30) Priority: 25.07.1998 US 122615 

(71) Applicant: TEKTRONIX, INC. 
Wllsonvllle, Oregon 97070-1000 (US) 

(72) Inventors: 

• Edwards, John W. 

Lake Oswego, Oregon 97035 (US) 



• Maurer, Steven D. 
Aloha, Oregon 97007 (US) 

(74) Representative: Molyneaux, Martyn William 
Langner Parry 
52-54 High Holborn 
London WC1 V 6RR (GB) 

Remarks: 



Amended claims in accordance with Rule 86 (2) 
EPC. V ' 




(57) A picture quality assessment method based up- 
on spatial location with or without subsampling within 
the picture divides the video picture into a plurality of 
arbitrarily sized, geometric segments. A segment 
weighting map is generated that provides one value for 
"safe picture" segments of the video picture and another 
value for the other segments. Optionally one or more of 
the segments may be subsampled to reduce processing 
time for realtime applications. The video picture on a 
segment basis is compared with a corresponding test 
video picture using a picture quality assessment algo- 
rithm to generate a quality error value for each segment. 
The quality error values are weighted according to the 
segment weighting map, and on an error display the 
weighted quality error values are displayed. 
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Description 

BACKGROUND OF THE INVENTION 

[0001] The present invention relates to the test and 
measurement of video image signals, and more partic- 
ularly to video picture quality assessment using spatial 
location with or without sub-sampling. 
[0002] Both subjective and objective measurements 
of video quality degradation in video image signals have 
been done. In subjective measurements human sub- 
jects visually compare and rate the degradation be- 
tween an original and an impaired video image signal- 
Objective quality measurements are done by comparing 
an impaired video image signal to its original video im- 
age signal. The comparison includes such processing 
as transforming video image signals into bit-map form, 
adjusting mutual amplitude gain, offset and spatial shift, 
filtering them in some fashion, subtracting them from 
each other, then filtering some more and representing 
the results either as error maps or as statistics derived 
from these maps. 

[0003] Subjective measurement methods suffer from 
a lack of reproducibility and consistency. Objective 
methods are mechanistic and lack the ability to differen- 
tiate between areas where human observers pay atten- 
tion ("attentional factors") and those areas without much 
interest to humans. 

[0004] Copending U.S. Patent Application Serial No. 
08/896,912, filed July 17, 1997 by Bozidar Janko and 
John Edwards entitled "Attentional Maps in Objective 
Measurement of Video Quality Degradation", addresses 
content-based picture quality by describing a method of 
isolating a particular content element, such as a human 
face, and applying a greater quality weighting to such 
element. However this method requires either a manual 
input as to which elements in the picture should be 
weighted more or a special processing algorithm that 
makes such a determination for every picture. 
[0005] It is well known that certain portions of a video 
picture are more likely to carry important information 
than others. The video industry uses a so-called "safe 
picture" and "safe title" area inside which the important 
elements of a program are generally captured by a cam- 
era and in which titles are contained. Essentially the safe 
areas define the central portion of the screen, i.e., the 
periphery of the video image is outside the safe area. 
Also presently measurement of picture quality is a com- 
pute intensive operation that cannot be performed in re- 
al time. For certain applications, such as satellite trans- 
mission where it is desired to compress the various 
channels as much as possible without appreciable deg- 
radation so that the satellite may carry more channels, 
realtime processing is desireable. 
[0006] What is desired is a simpler method of picture 
quality assessment using spatial location with or without 
subsampling rather than attentional maps. 



BRIEF SUMMARY OF THE INVENTION 

[0007] Accordingly the present inventions provides 
picture quality assessment using spatial location with or 

5 without subsampling. A video sequence, either a refer- 
ence or from a device under test, is divided into a plu- 
rality of arbitrarily sized, geometric segments. A seg- 
ment weighting map is generated that provides a low 
threshold for "safe picture" segments of the video pic- 

10 ture and a high threshold for other segments. Weighting 
coefficients may be used instead of thresholds. Option- 
ally one or more of the segments may be subsampled 
to reduce the amount of processing required for realtime 
applications at the expense of measurement accuracy. 

15 The video picture, both reference and test, is captured, 
segmented, optionally subsampled and processed by 
an appropriate picture quality assessment algorithm, 
such as JND or PSNR for example, to generate a quality 
error value for each segment. The segment weighting 

20 map is applied to each segment, and the result is dis- 
played as an error map. 

[0008] The objects, advantages and other novel fea- 
tures of the present invention are apparent from the fol- 
lowing detailed description when read in conjunction 
25 with the appended claims and attached drawing. 

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF 
THE DRAWING 

30 [0009] Fig. 1 is a block diagram view of a test and 
measurement instrument for performing picture quality 
assessment according to the present invention. 
[0010] Fig. 2 is a segment weighting map for picture 
quality assessment according to the present invention 

35 having threshold levels. 

[0011] Fig. 3 is a diagram of measured quality errors 
for each segment of the segment weighting map of Fig. 
2. 

[0012] Fig. 4 is an error display using the segment 
40 weighting map of Fig. 2 for the measured quality errors 
of Fig. 3. 

[0013] Fig. 5 is a segment weighting map for picture 
quality assessment according to the present invention 
having weighting coefficients. 
45 [0014] Fig. 6 is a flow diagram view for picture quality 
assessment according to the present invention. 

DETAILED DESCRIPTION OF THE INVENTION 

50 [0015] Referring now to Fig. 1 a test and measure- 
ment instrument 10 is shown based upon a personal 
computer interface (PCI) architecture. A personal com- 
puter (PC) 12 has a central processing unit (CPU) 14 
that includes a random access memory 16. The PC 12 

55 typically includes a floppy disc drive 18, a CD-ROM drive 
20, a hard disc drive 21 and an interface port 22 to which, 
for example, a mouse, a keyboard, a printer and the like 
may be coupled. The PC 12 is coupled to a PCI bus 24 
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to which other PCI compatible plug-in cards may be cou- 
pled. For the present instrument 10 a digital signal 
processing (DSP) board may be plugged into the PCI 
bus 24, which board includes a capture/analysis section 
26 and an optional generator section 28. Also coupled 
to the PCI bus 24 may be an encoder 30 for converting 
serial digital video to composite video and/or a decoder 
32 for converting composite video to serial digital video. 
[0016] The DSP board includes in the capture/analy- 
sis section 26 a random access memory (SDRAM) 34 
that contains a test sequence file 36 and a capture se- 
quence file 38. The generator section 28 also includes 
an SDRAM 40 that contains a generated test sequence 
file 42. In operation a test sequence may either be cap- 
tured from a source and stored in the test sequence file 
36 or may be generated from the generated test se- 
quence file 42. A video image sequence corresponding 
to the video test sequence that is received from a system 
under test to which the video test sequence is input Is 
captured and stored in the capture sequence file. The 
DSP then compares the test and capture files according 
to the desired measurement, such as just noticable dif- 
ference (JND) or power signal to noise ratio (PSNR) 
measurements or other picture quality assessment 
(PQA) algorithm. The resulting measurements are pro- 
vided as an output for display. 

[0017] The generated video test sequence may be in- 
put to the encoder 30 to produce a composite video sig- 
nal for display on an appropriate monitor. 
[0018] As shown in Fig. 2 each video image may be 
broken up into a plurality of geometric segments. For 
convenience rectangular segments are shown. The 
segments form a matrix of rectangles with arbitrary hor- 
izontal and vertical dimensions. A picture quality metric 
is computed for each segment using a picture quality 
assessment measure. A quality metric threshold, or 
weight, is assigned for each segment, so that for exam- 
ple the center segments receive a low error threshold 
value (high weight), such as 1, while those around the 
periphery receive greater error threshold values (low 
weight), to form a segment weighting map. The error val- 
ues generated by the comparison of the test sequence 
file with the capture sequence file are compared with the 
corresponding threshold values defined by the segment 
weighting map. As an example some segment error val- 
ues are shown in Fig. 3. Where the error value is less 
than the threshold in the segment weighting map, the 
resulting section in an error display is shown as having 
zero errors, as shown in Fig. 4. If the error value is equal 
to or greater than the threshold, then the error value is 
provided on the display for each segment. A segment 
weighting map with weighting coefficients is shown al- 
ternatively in Fig. 5. 

[0019] Although rectangular segments are described 
for ease of explanation, any arbitrarily sized, geometric 
picture segment may be used and arbitrarily located on 
the image. The threshold weighting described above is 
applied to such segments. The picture quality assess- 



ment measures alternatively may be multiplied by the 
weights indicated prior to applying the threshold values 
or in lieu of the threshold values. 
[0020] To decrease processing time for realtime ap- 
5 plications, the individual segments optionally may sep- 
arately or collectively be subsampled before being input 
to the picture quality assessment algorithm (for both the 
reference and test segments). Subsampling is defined 
as intentionally discarding pixels and/or lines in a set 
io pattern from the original image or segment. Any pattern 
may be used. For example, a simplistic 4:1 subsample 
scheme may be organized by discarding every other 
pixel and every other line. By providing both weighting 
after the PQA algorithm and subsampling before the 
15 PQA algorithm on a segment by segment basts, an op- 
erator may tune for the best balance between them for 
greatest accuracy while achieving realtime results. Sub- 
sampling achieves a similar, but different, effect as 
weighting, acting more like a reduction of viewing angle 
20 ~ moving back from the viewing screen. This presents 
a significant savings to the computation of the appropri- 
ate picture quality measure at a cost of being significant- 
ly more inaccurate when the subsample ratio gets too 
high. 

25 [0021] The total process is shown in Fig. 6. The ref- 
erence video sequence and test video sequence from 
the device under test are both captured in step 44 and 
stored in their respective files 36, 38. For this example 
two segments are shown, interior segment A which 
30 might correspond to an SDTV portion of an HDTV video 
sequence and surrounding segment B. The video se- 
quences are segmented at step 46. In this example only 
segment A is optionally segmented at step 48. In step 
50 the desired PQA algorithim is applied to each seg- 
35 ment, and in step 52 the segment weighting map is ap- 
plied to the PQA measures. The resulting error values 
for each segment are combined for display into the error 
display map at step 54. 

[0022] Thus the present invention provides picture 
40 quality assessment by dividing the video picture into ar- 
bitrary geometric segments, optionally subsampling 
each segment, finding the quality error for each seg- 
ment, and comparing the error with a segment weighting 
map where specified segments have a low threshold 
45 value (high weight) and the other segments have a high- 
er threshold value (low weight) to produce an error dis- 
play. 



50 Claims 



A method of assessing video picture quality com- 
prising the steps of: 

dividing a video picture into a plurality of geo- 
metric segments; 

generating a segment weighting map having a 
specified quality metric threshold for each seg- 
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ment; 

comparing the video picture with a correspond- 
ing test video picture to produce for each seg- 
ment a quality error value; and 
displaying the quality error value for each seg- 5 
ment where the quality error value equals or ex- 
ceeds the specified quality metric threshold 
from the segment weighting map. 

The method as recited in claim 1 further comprising 10 
the step of selectively subsampling one or more of 
the geometric segments for the video picture and 
corresponding test video picture prior to the com- 
paring step. 

15 

A method of assessing video picture quality com- 
prising the steps of: 

capturing a reference video sequence and a 
corresponding output video sequence from a 20 
device under test; 

dividing the reference video sequence and out- 
put video sequence into a plurality of corre- 
sponding geometric segments; 
generating a segment weighting map having a 25 
specified value for each geometric segment; 
comparing the reference video sequence with 
the output video sequence on a segment by 
segment basis using a picture quality assess- 
ment algorithm to produce a quality error value 30 
for each geometric segment; 
weighting each quality error value according to 
the segment weighting map to produce weight- 
ed quality error values; and 

displaying the weighted quality error values on 35 
an error display map for each geometric seg- 
ment 

The method as recited in claim 3 further comprising 
the step of selectively subsampling one or more of *o 
the corresponding geometric segments from the 
reference video sequence and the output video se- 
quence prior to the comparing step. 

45 
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